Field
The following disclosure relates generally to light-field imaging and light-field image processing.
Description of the Related Art
The spatial world is three-dimensional, yet conventional photographs record only a two-dimensional image. Thus, conventional photographic cameras reduce the complex interplay of light and matter in the three-dimensional world to a flat, two-dimensional recording of light intensity, as detected from the object space within the field of view of the camera. This flattening effect is a result of imaging, in which light rays reflected and/or scattered at different points on an object within the field of view of the camera are focused by a lens to corresponding points on an image plane. Angular information is lost in this process; for example, the light intensity recorded at a given pixel in a conventional image does not indicate the respective intensity contributions of light rays that originate from the corresponding point in the field of view with different angular orientations. Instead, the intensity measured at each point in the image plane is indicative of the combined intensity of the various light rays that enter the camera with different angular orientations from the corresponding point in the field of view. Thus, various properties like depth and certain light statistics cannot be determined quantitatively from a conventional image.
The flattening from three dimensions to two dimensions in a conventional camera significantly limits the information content of the image. Perhaps the simplest consequence of this flattening is ambiguity in depth, with objects behind and in front of the focal plane blurred (out of focus) in ways that can be difficult to describe quantitatively. There have been several methods developed to acquire depth information, which typically supplement a single two-dimensional image with at least one other measurement that can map to the third dimension. These include time gating (radar), interference of multiple beams (holography), axial displacement (focal stacks, phase retrieval), and transverse displacement (stereoscopic vision, lenslet arrays). Each method has its associated costs, usually in both physical and computational complexity. Typically, there are trade-offs in the final image quality as well. In fields that require accurate measurements, such as scientific, medical, and military imaging, the loss of three-dimensional data of the object space is disadvantageous. In such fields, and many others, the ability to record both spatial and angular information with high resolution would be advantageous.
One method of obtaining information regarding the respective intensities of light rays with different angular orientations from within the field of view is to provide a wavefront sensor, such as a Shack-Hartman array of lenslets in proximity to a sensor (e.g., a CCD or CMOS sensor). Each lenslet samples a spatially localized region of the wavefronts of light that enter the instrument from the field of view, and allows local angular information to be recorded on the sensor. In this way, the sensor can detect the respective intensity of light rays that arrive at each lenslet from different angular directions. This four-dimensional information of light intensity at each position (x, y) for each angle (θx, θy) quantifies the light field within the instrument's field of view.
The price paid to acquire angular information using such lenslets is a reduction in resolution: spatial sampling is determined by the size of the lenslets rather than the size of the pixels in the camera sensor. Several pixels in the camera sensor correspond to each lenslet and now record the light intensity for different ray angles, so the information is distributed in phase space. This fundamental trade-off in spatial vs. angular resolution has plagued lenslet imaging since Lippman introduced lenslet imaging in 1908.